i think the [first post](<./circuit utility.md>) on circuit utility was overexplaining and similar ideas were already captured by existing things like [synthsaebench](https://decoderesearch.github.io/SAELens/latest/synth_sae_bench/), [this p metric from FSEC](https://www.lesswrong.com/posts/uDrsffSLzWD6cDnTt/evidence-for-feature-specific-error-correction-in-llms-1#Superellipse_exponents_across_direction_types), and [measuring narrow task success from logits given sample size](https://www.lesswrong.com/posts/fSRg5qs9TPbNy3sm5/deep-causal-transcoding-a-framework-for-mechanistically#Case_Study__Learning_Jailbreak_Vectors). this "update" is more like: i have went about a particular method that would probably score well on general versions of those metrics

but after touching ground with reality a bit wrt DCT's I have been fairly relieved at results I've been getting with different intuitions. for example, you can basically investigate feature circuits from dcts (exp ones, anyways) by taking the pullback of an output factor, and mean-ablating a combination of top activating input factors.

this seems to fall short of causal scrubbing because feature level hypotheses are naively interactions with vectors on the unit sphere, which is infinitely more expensive to validate than component level hypotheses (however, this comparison now [seems a bit dubious](#appendix) to me). however, converting arbitrary feature circuits into component-level circuits still seems like a very useful thing to have, even with lossiness, and pullbacks still yield non-trivial data-free info

pullbacks also often show circuits that arent sparse one-to-one connections. ranking output factors $V^{\ast}$ by propensity to upweight a certain logit does not necessarily produce $u_i \rightarrow v_i$ where i is the index of a high ranked output factor, and possibly not even many-to-one maps of this sort

there's still quite a bit of different things to do in this direction, like

 -  what other useful things can be done on DCTs to yield faithful shallow circuits?
	- how to avoid reinventing existing issues by introducing reconstruction error or sparsity or excessive data dependence
	- input dependent info?
	- work with other attribution methods (integrated gradients, eap-ig)
- quantifying circuit generalization (like causal scrubbing) still seems very important for identifying independent(-up-to-some-error) events, simplifying probability expectations a lot
	- can DCTs simplify [interactions with prior states](https://www.lesswrong.com/posts/jqnda7W9hugFP4Cnr/obstacles-in-arc-s-agenda-low-probability-estimation#Does_LPE_work_at_all_), maybe solving a large chunk of issues that befall lpe as ~pursued currently
- try simpler nets as dct is a very generalizable method
- check if DCTs actually provide meaningful advantage over existing methods for identifying some expectation of a behavior

i suspect DCTs are more likely to identify faithful computations because the approach is not fundamentally constrained by having to [represent the input data structure](https://www.lesswrong.com/posts/gYfpPbww3wQRaxAFD/activation-space-interpretability-may-be-doomed). it is less obvious that the approach can always efficiently and adversarially uncover circuits formed for acting on a tiny fraction of dataset/input space, and you may end up with similar streetlight effects to other approaches because you still need to specify *some* examples, but current evidence so far has shown that DCTs can generalize non-trivially. 

an obvious recurring issue is that deep causal transcoding is not transcoding per se (DCTs cannot produce factors that replace computation in a slice)

... which is why it would be convenient if factors formed a useful abstraction of a transformer slice, as this would highly simplify [edge-attribution patching](https://arxiv.org/pdf/2310.10348#page=3) and other such methods. currently ive seen non-trivial interactions like many-to-one feature circuits, where pulling some ranked output feature back to input features and mean-ablating the top activating inputs causes similar behavioral changes, compared to steering on output feats directly. assuming input feats exist in a certain layer and inhibit/trigger output feats this feels pretty similar to the theory of how DCTs work (which should've been obvious probably)

however, the causal part of the objective[^1] used to train exponential DCTs doesn't explicitly consider factoring features into circuits, and DCTs do not explicitly produce an MLP with a proposed activation function. it is easy to observe logit differences by steering/ablating feats on the lm, but doing EAP this way might be too weak or not resemble a 'causal attribution' enough... maybe you could just do EAP by zeroing out all except some relevant $v_l$ s and having your metric be the causal part? big if true

![434](<./Pasted image 20260816234811.png>)

👀

however, what we do have here *are* already circuits. arguably, these types of circuits still seem anchored in a narrow case of elicitation; doesn't it seem bad to deduce estimates with only a simple understanding of the components that cause outputs and behaviors, as DCTs merely model shallow circuits? i think this is not obviously true

- shallow circuits are perfectly compatible with [how](https://www.lesswrong.com/posts/azCuKGJTecCDLBEEj/maybe-a-bag-of-heuristics-is-all-there-is-and-a-bag-of) [llms](https://www.lesswrong.com/posts/fSRg5qs9TPbNy3sm5/deep-causal-transcoding-a-framework-for-mechanistically#Theory) [work](https://www.lesswrong.com/posts/a9KqqgjN8gc3Mzzkh/llms-as-giant-lookup-tables-of-shallow-circuits) (however this could be less plausible for other nns not along [the current imitative learning trend](https://www.beren.io/2025-08-02-Most-Algorithmic-Progress-is-Data-Progress/))
- uncovering as broad a range of shallow circuits on a task is still a great amount of detail, even more if you can assign independent metric attributions to every circuit or group of circuits
- this particular class of approaches may be less worse for recovering useful forms of the 'true' model units (compared to transcoder circuits or cross-layer transcoders[^2]); DCTs being as sample-efficient as they are, they could do better at finding a broader worst-case range of possible model computations

in the other direction, extracting more details out of circuits implemented in DCT factors seems pointless if you're just going to sample over a training distribution, and you might as well train a bunch of DCTs, take the top $k$ ranking output features for every DCT, and sample for them over a training distribution. i do think there is a kernel of truth to this

- one assumption that makes ARC's example of mechanistic understanding compelling is that features of the model *and data* are i.i.d. not having identical probabilities probably makes estimation on this route meaningfully harder
- it has not been clear how this approach handles statistically understanding the input distribution, but i think there is a lot to learn from how SAEs learn features of the data, and also how shallow circuits -- observed across multiple DCTs, or learned in 2 or more layers -- produce certain outcomes from 'semantic summaries' of the input 

[^1]: $\sum_{l} \langle u_l, \Delta^{s \rightarrow t}_R(v_l) \rangle$

[^2]: both are likely underperforming here for aforementioned data-dependence reasons, and CLTs are  [unfaithful](https://www.lesswrong.com/posts/6CS2NDmoLCFcEJMor/cross-layer-transcoders-are-incentivized-to-learn-unfaithful)
## toy constructions

i have wondered if this would be useful for validating the method's sucess. mixed thoughts on this though. this class of methods attempts to fix a problem that is hard specifically because there are many noisy [generalizations](https://www.lesswrong.com/posts/8ms977XZ2uJ4LnwSR/decomposing-independent-generalizations-in-neural-networks), which makes it hard to design a decent toy model (for me anyways)

one example ive floated around is something like training a transformer to implement two functions -- or 'generalizations' -- and then bringing down the expectation of one of the uniformly learned generalizations by plain sampling or something else (not including DCTs themselves). this seems almost too trivial though, almost a mismatch as DCTs have basically n
## appendix 

i have often mentioned causal scrubbing as an effective method to throw in the mix but i mean it hypothetically or regarding [the rigor of the metric](https://www.lesswrong.com/posts/JvZhhzycHu2Yd57RN/causal-scrubbing-a-method-for-rigorously-testing?commentId=oikvsKEmmBrBr9Yno), since otherwise there are pretty large gaps in literally implementing that method here

- CaSc supposes an explanation that is specified enough, and doesn't validate explanations on a feature-basis, and applying such a metric here seems like a mismatch (and especially for shallow feature circuits). supposing that transformer lms actually can be modeled as an  ensemble of shallow circuits, and supposing all the circuits that elicit a behavior at a transformer slice were found, still feels like a mismatch
- circuits here are being identified from a slice of a network and you could find salient circuits mediating similar behaviors across many possible slices. that is, there should be more done to have reasonable assurance that, for a circuit, the range of layers for some behavior mediated from an input token/latent variable is more... correct[^3]. thus further ensuring that some circuit is an independent event
	- to do this, you'd need to consider more detail, like attribution with respect to the residual stream across the entire model, or attribution w.r.t DCT factors learned from earlier slices (and doing this with various permutations), or inherently learn factors $U, V, W, ...$ and compute objective scores between pairs or permutations (and either using layer index averages or some calibration procedure for these intermediate computations). attribution through the whole model seems pretty intractable to me
	- ![200](<./Pasted image 20260819132233.png>)
	- ![200](<./Pasted image 20260819224226.png>)
	- this is a sketch off the top of my head, but it might be underestimating the extent of sparsity between connections[^4]; sparsity would naturally simplify finding expectations over the model, but even if there isn't much sparsity, we can probably just rank circuits by attribution scores, and throw out input connections that aren't that important in the estimation (attribution across DCTs could be quite more complicated though)
- it's hard to overstate how difficult it is to satisfy CaSc's metric. for reference, [this](https://www.lesswrong.com/posts/sTe78dNJDGywu9Dz6/solving-the-mechanistic-interpretability-challenges-eis-vii) and [this](https://www.lesswrong.com/posts/k43v47eQjaj6fY7LE/solving-the-mechanistic-interpretability-challenges-eis-vii-1) were among few explanations that attained >90% expected loss recovered. without more specificity i don't know why a shallow feature circuit should attain higher than 50% as an explanation without applying EAP, or other aforementioned takes.
- obviously the issue of considering a network that can generalize out of distribution and you are specifically in a situation where you don't have an extensive input distribution 
- the naive approach to validating hypotheses by replacing all other model-native features with random unit vectors supposes:
	1. that we can just find and ablate/edit them like components (if we could this would've made DCTs and [other work](https://www.lesswrong.com/posts/LajDyGyiyX8DNNsuF/interim-research-report-activation-plateaus-and-sensitive-1) moot)
	2. some kind of pre-CIS mindset that doesn't take advantage of models privileging certain directions or performing [error-correction](https://www.lesswrong.com/posts/uDrsffSLzWD6cDnTt/evidence-for-feature-specific-error-correction-in-llms-1) otherwise

from what i've gathered, aiming for a metric with CaSc's rigor here also seems dubious but attribution as a way to measure circuit independence is generally not, but is obviously more prone to being inexact. the extent that this would underperforms sampling is unclear

[^3]: i think it is wrong to focus on broad ranges of potential model computations while also not considering comparatively more plausible computational pathways stemming from input tokens

[^4]: naturally there are few dots so i feel inclined to draw a lot of lines, but with many times more factors, i expect way more output feats attributed to unique input feats